Conventionally, a game operating apparatus which includes a longitudinal housing having a holding portion to be wrapped and held with a palm of a user, and a game system using such a game operation apparatus, have been known. This operation apparatus is provided with an imaging section at a front end, in the longitudinal direction, of the housing. In the game system, an LED module that emits infrared light (infrared ray) is arranged in the vicinity of a screen of a display. The operating state of the game operating apparatus can be estimated by, for example, capturing an image of the infrared light outputted from the LED module by using the imaging section, and analyzing the position of the infrared light in the captured image.
The game system as described above adopts the configuration in which the LED module that emits infrared light is arranged in the vicinity of the display in order to estimate the operating state of the operation apparatus. Regarding this point, the inventors have realized that the configuration of the game system can be made simpler to achieve the same function as described above.
Therefore, it is an object of the exemplary embodiments to provide an information processing system capable of detecting the position and/or orientation of an operation apparatus with a simple configuration.
In order to attain the object described above, the following configurations are exemplified.
A configuration example is an information processing system including at least a reflection member, a hand-held information processing apparatus capable of image processing, and an information processing apparatus communicable with the hand-held information processing apparatus. The reflection member is a retroreflecting material part. The hand-held information processing apparatus includes an imaging section, a light emitting section, a light source detecting section, a calculator, and an output section. The imaging section captures an image. The light emitting section emits light. The light source detecting section detects, in the image captured by the imaging section, light source images including an image of the retroreflecting material. The image of the retroreflecting material indicates light that has been emitted from the light emitting section and reflected by the retroreflecting material part. The calculator calculates positions, in the captured image, of the detected light source images. The output section outputs positional information indicating the calculated positions. The information processing apparatus includes an image generator that estimates an orientation or a position of the hand-held information processing apparatus on the basis of the positional information outputted from the output section, and generates an image on the basis of a result of the estimation. The light source images are image portions, in the captured image, relating to pixel groups corresponding to various types of reflected lights. The image of the retroreflecting material is an image portion, in the captured image, relating to a pixel group corresponding to the reflected light from the retroreflecting material part.
According to the above configuration example, it is possible to provide an information processing system capable of recognizing the direction to which the hand-held information processing apparatus is turned, the orientation and/or position of the hand-held information processing apparatus, etc., with a simple configuration.
In another configuration example, the information processing system may further include a specifying section that specifies the image of the retroreflecting material in the captured image on the basis of the positional information of the light source images calculated by the calculator. Further, the specifying section may calculate an interval between the light source images on the basis of the positional information of the light source images, and specify the image of the retroreflecting material on the basis of the interval between the light source images. Alternatively, the specifying section may specify the image of the retroreflecting material on the basis of a positional relationship of the light source images. Still alternatively, the specifying section may specify the image of the retroreflecting material on the basis of whether or not the positional relationship is a positional relationship in which the light source images are arrayed in a straight line.
According to the above configuration example, it is possible to detect the reflected light from the reflection member by low-load processing.
In another configuration example, the information processing system includes an imaging target member having a plurality of the retroreflecting material parts. In the imaging target member, the plurality of the retroreflecting material parts may be arranged at equal arrangement intervals. On the basis of the positional information of the positions, in the captured image, of the light source images calculated by the calculator, a plurality of sets of two light source images, each having the same interval between the two light source images, may be specified as images of the retroreflecting material corresponding to the plurality of the retroreflecting material parts included in the imaging target member.
According to the above configuration example, it is possible to detect the direction to which the hand-held information processing apparatus is turned and/or the orientation of the hand-held information processing apparatus by low-load processing while simplifying the configuration of the system.
In another configuration example, the calculator may further calculate brightnesses of the detected light source images or sizes thereof in the captured image. Further, the information processing system may include a plurality of types of imaging target members each having a plurality of the retroreflecting material parts. In each of the plurality of types of imaging target members, the retroreflecting material parts are disposed at different positions. The calculator may distinguish the types of the imaging target members on the basis of the distance between the detected light source images. The information processing system may further include a specifying section that specifies the image of the retroreflecting material in the captured image, on the basis of the brightnesses or sizes of the light source images.
According to the above configuration example, it is possible to execute information processing in which the plurality of imaging target members are property used. Further, the reflected light from the retroreflecting material part can be detected by utilizing the characteristics of the light source, thereby reducing the processing load.
In another configuration example, the image generator may generate an image including at least an indication object indicating that a position on a screen is indicated.
According to the above configuration example, when a user is operating the hand-held information processing apparatus, the user is allowed to intuitively understand the position, on the screen, corresponding to the operation.
In another configuration example, the light emitting section may emit infrared light as the light.
According to the above configuration example, viewing action of the user is not hindered. Further, even when the room is dark, the direction to which the hand-held information processing apparatus is turned and/or the orientation of the hand-held information processing apparatus can be recognized.
In another configuration example, the information processing system includes a plurality of the retroreflecting material parts, and the plurality of the retroreflecting material parts may be separately disposed in least three positions. Further, the retroreflecting material parts located in three positions may be arrayed in a straight line. In addition, the retroreflecting material parts located in three positions may be arrayed at substantially equal arrangement intervals.
According to the above configuration example, it is possible to improve the detection accuracy of the direction to which the hand-held information processing apparatus is turned and/or the orientation of the hand-held information processing apparatus, while simplifying the configuration of the system.
In another configuration example, the hand-held information processing apparatus may further include a wireless communication section that transmits the positional information of the light source images in the captured image to the information processing apparatus.
According to the above configuration example, it is possible to provide a user-friendly information processing system.
In another configuration example, the information processing system includes a flickering determining section that determines presence/absence of flickering of each light source image, on the basis of change in brightness of the light source image in the captured image over a plurality of frames. The calculator may exclude, from the processing target, a light source image that is determined by the flickering determining section to be flickering.
According to the above configuration example, the processing load on the detection process can be reduced. Further, the detection accuracy of the image of the retroreflecting material can be improved.
In another configuration example, the light emitting section may be provided at a position near the imaging section.
According to the above configuration example, the light emitted from the light emitting section can be easily and reliably captured by the imaging section.
In another configuration example, the information processing apparatus may be a stationary game machine.
According to the above configuration example, by using the hand-held information processing apparatus as an operation apparatus, for example, it is possible to provide a game system, with a simpler configuration, capable of executing game processing that utilizes motion of the operation apparatus itself.
Another configuration example is an information processing system including at least a reflection member and a hand-held information processing apparatus capable of image processing. The reflection member is a retroreflecting material part. The hand-held information processing apparatus includes an imaging section and a light emitting section. The imaging section captures an image. The light emitting section emits light. The information processing system further includes a light source detecting section, a calculator, and an image generator. The light source detecting section detects, in the image captured by the imaging section, light source images including an image of the retroreflecting material. The image of the retroreflecting material indicates light that has been emitted from the light emitting section and reflected by the retroreflecting material part. The calculator calculates positions, in the captured image, of the detected light source images. The image generator estimates an orientation or a position of the hand-held information processing apparatus on the basis of the positional information indicating the calculated positions, and generates an image on the basis of a result of the estimation.
According to the above configuration example, it is possible to provide an information processing system capable of recognizing the position and/or orientation of the hand-held information processing apparatus, with a simple configuration.
According to the present embodiment, it is possible to provide an information processing system capable of detecting the position and/or orientation of a hand-held information processing apparatus which is an operation apparatus, for example, with a simple configuration.